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The purpose of this research was to determine 
how infonnation synthesis skills can be taught 
effectively, and to discover how the level of 
synthesis in student writing can be effectively 
measured. The intervention was an infonnation 
synthesis lesson that broke down the synthesis 
process into sequenced tasks. Researchers 
created a rubric which they used to assess 
students’ levels of information synthesis 
demonstrated in their final research essays. A 
form of counting analysis was also created to 
see if other methods could help in measuring 
synthesis. 

Findings from the rubric analysis revealed that 
students appear to benefit from the synthesis 
lesson. The level of synthesis, however, 
remains low overall. In addition, the study 
showed that the different measures of synthesis 
established were able to identify different 
levels of information integration. Discovering 
effective ways to measure and teach synthesis 
continues to be essential in helping students 
become infonnation literate. 
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Introduction 

Perhaps the most essential, and certainly 
one of the most complex research skills, is 
the ability to synthesize information. One 
researcher, J.D. Johnson (2009) writes: 
the ability for people to assimilate 
information they find into coherent personal 
strategies is perhaps the critical modern 
survival skill” (p. 601). Information 

synthesis is the process of analyzing and 
evaluating infonnation from various 
sources, making connections between the 
information found, and combining the 
recently acquired information with prior 
knowledge to create something new. 
Infonnation synthesis strategies are essential 
skills. Without them, we cannot derive new 
knowledge from these large amounts of data 
(Larsen, Wactlar & Friedlander, 2003; 
National Science Board, 2005). Effective 
information synthesis is also vital in 
developing effective writing and 
communication skills to share new 
knowledge. Coherent information synthesis 
is, therefore, required to productively 
participate in and contribute to our 
information-rich society. Yet college 
students have difficulty analyzing and 
synthesizing different pieces of information 
(Howard, Serviss, & Rodrigue, 2010). 

The research questions explored in this 
study ask whether information synthesis 
skills can be taught effectively by 
scaffolding this complex cognitive task, and 
how the level of synthesis in student writing 
can be effectively measured. The study 
described here investigates an information 
synthesis lesson given to students in a 
university English writing class. The lesson 
broke down the synthesis process into 
several stages requiring students to go 
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through infonnation synthesis in discrete 
steps by means of individual reading and 
group discussion. At the end of the lesson 
students were asked to write a synthesized 
paragraph. The authors also collected final 
research papers in the class. Both writing 
products were analyzed for evidence of 
synthesis to evaluate the impact of the 
synthesis lesson as well as to establish 
different metrics for measuring synthesis. 

Findings from the study revealed that 
students appear to benefit from the synthesis 
lesson. There were more instances of 
information synthesis in the final papers of 
students who received the lesson. The level 
of synthesis however, remained low overall. 
In addition, studies measuring synthesis 
identified different levels of information 
integration. The synthesis rubric used in this 
study reaffirms that synthesis, and the 
assessment of it, includes numerous skills 
and competencies. The implications of these 
findings suggest that teaching synthesis 
through scaffolding this process requires 
more than a single lesson and should 
perhaps be provided early in the semester. 
Rubrics and additional metrics that identify 
synthesis can be used to communicate to 
students about certain features of 
synthesized papers and can help instructors 
and librarians to more accurately assess 
student work and provide them meaningful 
feedback for improvement. 

Problem Statement 

The literature on infonnation seeking 
behavior shows that students have 
superficial information seeking and research 
skills (Asher, Duke, & Green, 2010; 
Fitzgerald, 2004; Head and Eisenberg, 
2009; Kolowich, 2010). In one study only 
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50% of student participants were able to 
successfully synthesize information from 
multiple sources (Mateos & Sole, 2009). 
Infonnation synthesis is a key skill for 
participants in our knowledge society and 
requires complex processing (Fitzgerald, 
2004; Goldman, 2004). Yet information 
literacy instruction and practice tend to 
favor easily-defined skills that often only 
emphasize the search component of the 
research process, leaving out higher order 
processes like information synthesis (Lloyd, 
2007; Montiel-Overall, 2007; Simmons, 
2005; Tuominen, Savolainen, & Talja, 
2005; Webber & Johnston, 2000). Similarly, 
in the writing classroom, teachers are 
largely unfamiliar with how to teach 
synthesis sometimes implying it is a linear 
process (McGinley, 1992), leading Mateos 
and Sole (2009) to call for a “unique, 
careful teaching approach” (p. 448). In 
response, this study seeks to address the 
question of how to teach students to 
effectively synthesize infonnation from 
multiple sources, and how to effectively 
assess and identify synthesis when it occurs 
in student work. 

Literature Review 

Infonnation synthesis appears in the 
literature under several different guises. 
Commonly used tenninology to describe the 
process of analyzing and evaluating 
information from various sources is multiple 
document (or source) comprehension 
(Goldman, Braasch, Wiley, Graesser, & 
Brodowinska, 2012; Goldman & 
Scardamalia, 2013), multiple document 
processing (Goldman & Scardamalia, 2013), 
transliteracy (Andretta, 2009; Thomas et al., 
2008), intertextuality (Goldman, 2004; 
Stadtler & Bromine, 2013), writing from 
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sources (Howard, Serviss, & Rodrigue, 
2010), and information synthesis (Blake & 
Pratt, 2002; Goldschmidt, 1986). Relevant 
literature to this study can be found in 
library and information science (information 
literacy and information problem solving 
models), education (cognition and literacy 
instruction), and composition and rhetoric 
(writing). Each field approaches the subject 
from a different angle. Different research 
and resulting instructional approaches of the 
various fields are discussed below. 

Information Problem Solving Models 
Infonnation problem solving models, also 
known as information literacy models, 
mostly serve as scaffolds for teaching the 
research process or as frameworks when 
studying the same process. Information 
synthesis appears in all of the most well- 
known models. The The Big 6 model is an 
information problem solving model 
developed by Eisenberg and Berkowitz, and 
is used widely in K-12 schools. This model 
includes synthesis as step five in their six- 
step stages, which also includes task 
definition, infonnation seeking strategies, 
location and access, use of information, and 
evaluation (Eisenberg & Berkowitz, 1990; 
Lowe & Eisenberg, 2005). This particular 
step includes organizing from multiple 
sources and then presenting the information. 
Here students are directed to read and then 
write from their notes from previous steps 
and to reflect on how best to present this 
information. In Stripling's six-phase Model 
of Inquiry, synthesis is contained in the 
construct phase (Stripling, 2010). This is 
where the bridge is built from previous 
knowledge to draw new conclusions, where 
conflicting information is confronted, 
conclusions are drawn, and evidence-based 
opinions are formed. In the Information 
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Search Process (ISP), Kuhlthau does not use 
the label of synthesis, but does include the 
process itself in the formulation stage 
(Kuhlthau, 1991; Kuhlthau, Heinstrom, & 
Todd, 2008). The formulation stage is the 
fourth of six stages, where the learner forms 
a focus from all the information 
encountered, identifying and selecting ideas 
from multiple sources to form a focused 
perspective. 

Information synthesis, a higher form 
of educational thinking 
Infonnation synthesis most commonly 
appears in the education literature as a level 
in the original Bloom’s Taxonomy of 
Learning Domains (Bloom, 1956). This 
Taxonomy is a classification for 
understanding student learning and to 
promote higher fonns of educational 
thinking. The Taxonomy is often depicted 
as a pyramid with the higher fonns of 
thinking at the top. For understanding the 
cognitive domain of learning, the Taxonomy 
builds upon steps beginning with factual 
knowledge and moving to comprehension, 
application, analysis, synthesis, and 
evaluation. Synthesis, ranked second from 
the top in the original Taxonomy, is 
considered one of the most important goals 
in the field of education. Here, synthesis is 
defined as the building of structures or 
patterns from a variety of elements with 
emphasis on creating some new meaning or 
a new structure from the elements. Some of 
the keywords involved in synthesis include: 
combine, create, design, and summarize. In 
Krathwohl’s (2002) revision of Bloom’s 
Taxonomy, the original categories were 
renamed and their definitions revised to 
represent more active thinking. Synthesis 
was renamed to “create” and changed places 
with “evaluation” as the top category in the 
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domain. The create category is defined as 
putting together elements to make a whole, 
including the elements of generating, 
planning, and producing. 

The role of synthesis in text 
comprehension 

Infonnation synthesis can be seen in the 
area of text comprehension, specifically in 
multiple-source comprehension (also known 
as multiple document processing, or 
intertextuality). Historically, text 
comprehension research involved single¬ 
document comprehension; this was not 
extended to multiple texts until the 1990s 
when Wineburg (1991) studied how novices 
and experts reasoned about a historical 
event using multiple documents. Using 
think-aloud protocols Wineburg identified 
the strategies people used to come to a 
conclusion. More researchers followed (see 
Stadtler & Bromine, 2013 for details), 
resulting in a better understanding of the 
various strategies employed when 
processing multiple documents (e.g. 
Perfetti, Rouet, & Britt, 1999). According to 
Goldman and Scardamalia (2013), to be 
successful at synthesizing infonnation from 
multiple documents, students need to be 
taught content knowledge, source expertise, 
and an understanding of how knowledge is 
created in the field of study. Once these are 
in place students can evaluate infonnation, 
integrate it into existing belief structures, 
and create new knowledge. Both Jucks and 
Paus (2013) and Goldman and Scardamalia 
(2013) note the social aspect of creating 
meaning, and they emphasize the use of 
discussion when teaching multiple- 
document processing in general and the 
resolution of conflicting infonnation 
between documents. Based on this research, 
the current study incorporates group 
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discussion in the intervention, alternated by 
individual reading and reflection. 

Synthesis in the writing classroom 
The rhetoric and composition literature 
addresses how students learn to synthesize 
multiple texts. Synthesis is complex in 
nature; therefore, the reading processes, the 
writing processes, and writing from multiple 
sources are all relevant to this discussion. 

This literature also considers synthesis to be 
a cognitively demanding task (Mateos, 
Martin, Villalon & Luna, 2008), which 
requires multiple activities such as 
organization, comprehension, problem 
detection, and problem solving (Braten, I., 
Stromso, 2003). Similarly, Nelson and 
Hayes (1988) noted that in order to write 
from multiple sources, students had to 
“coordinate a number of supporting 
activities.” Flower, et al. (1990) determined 
that synthesis is a risky endeavor, where the 
reader’s experience and knowledge, the text, 
and “reality itself may resist synthesis” (p. 
50). Not surprisingly, only 50% of those 
high school and university students who 
were studied could successfully synthesize 
(Mateos & Sole, 2009). Torraco (2005) 
views synthesis as a creative activity “that 
produces a new model, conceptual 
framework, or other unique conception 
informed by the author’s intimate 
knowledge of the topic” (p. 362). The same 
author describes four forms of synthesis, 
including a research agenda, a taxonomy or 
classification construct, alternative models 
or conceptual frameworks, and meta¬ 
theories. 

Nancy Spivey (1989), a major contributor to 
research in this area, elaborates on the major 
components of discourse synthesis, 


Communications in Information Literacy 9(1), 2015 

including the ability to “select, organize and 
connect content from sources texts as they 
compose their own new texts” (p. 9). Like 
Wineburg (1991), both McGinley (1992) 
and Spivey (1984) note that much variation 
existed in students’ writing processes with 
proficient and non-proficient college readers 
making different decisions in the ways they 
chose to make connections between texts. 

This literature also focuses on how to 
effectively teach the synthesis process. 
Mateos and Sole (2009) found that very few 
teachers knew how to help students go 
beyond connecting main ideas between 
different sources, while McGinley (1992), 
looking at the connection between writing 
and thinking processes, concluded that 
teachers should avoid implying that writing 
from multiple sources is wholly linear. 
Following a collaborative approach similar 
to the current study, Fluellen (2011) paired 
students together to read aloud and map 
concepts. McGregor (2011) used coded, 
graphical representations of student work to 
start conversations about how students use 
sources. Another solution requires 
instructional technology to teach synthesis 
at younger stages, such as TurboCite or 
TurboWrite (Tooley, 2005). 

Measuring information synthesis 
The usefulness of using rubrics to help 
measure infonnation literacy skills has been 
well documented by Oakleaf and others. 
Oakleaf comments on the ability of a rubric 
to “capture useable data about information¬ 
seeking behavior,” and on the value of the 
rubric development process itself (2007, p. 
28). 

While no comprehensive rubric exclusively 
evaluating information synthesis was found, 
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numerous rubrics have been created that 
evaluate information literacy, and they 
typically include elements of synthesis. 
Commonly found aspects of synthesis are as 
follows: establishing associations between 
texts, recognizing patterns among 
information (similarities, differences, 
unique instances), organizing information to 
express these relationships and patterns by 
using transitional sentences, and other 
explicit or implicit markers. Relevant 
rubrics include the Inquiry and Analysis 
VALUE Rubric (Association of American 
Colleges and Universities [AACU], 2010), 
Northern Arizona University’s (NAU) 
Synthesis Essay Rubric, Rubric Assessment 
of Information Literacy Skills (RAILS) 
Using Infonnation to Accomplish a Purpose 
Rubric, the Evergreen Synthesis Paper 
Rubric (Ford), and the General Education 
Assessment Rubrics (Klassen, 2014). We 
detennined that combining and rethinking 
pieces of these current rubrics would best 
suit the task of detennining whether or not 
synthesis was present in student papers. 

The rubrics listed above and an in-depth 
analysis of numerous student papers 
informed the creation of the information 
synthesis rubric (see Appendix A) used in 
this study. The VALUE rubric was 
beneficial as it was developed by teams of 
faculty at colleges and universities across 
the United States. The aim of the VALUE 
project, which resulted in the creation of 16 
different rubrics, was to have a national 
framework to support common dialog and 
understanding in specific areas for 
undergraduate level work (AACU, 2010). 
The Inquiry Analysis rubric heavily 
informed a category (subscale D) in our 
rubric which focused on source 
organization. The Synthesis Essay Rubric 
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was a collaborative effort between NAU’s 
eLearning Center and faculty to design 
effective instruction and assessment. The 
Synthesis Essay Rubric was especially 
important in creating subscale C in the 
present study: identifying conversations. 
The Using Information to Accomplish a 
Purpose Rubric was created for a library 
instruction teaching workshop focused on 
assessment and deposited in the RAILS 
repository, a funded research project 
investigating the use of analytical rubrics in 
the assessment of infonnation literacy. 
Terry Ford, from Evergreen State College, 
created the Synthesis Paper Rubric. Ford’s 
rubric was helpful in providing vocabulary 
to describe different skill levels for the 
present study: emerging, developing, and 
proficient. Ford’s rubric was also used 
extensively to create subscale E: analyzing 
sources to create something new. Klassen’s 
General Education Assessment Rubrics 
(2014) helped to distinguish levels of 
progression as students become more 
proficient in developing a range of skills 
essential to general education. The rubric 
adapted two of the seven categories in the 
Synthesis Rubric, including information 
literacy skills and synthesis and critical 
thinking skill patterns. The researchers 
relied on these five rubrics to develop a 
rubric which more adequately provided a 
guide for identifying synthesis in student 
work. 

Research Design and 
Methods 

The research team included two faculty 
librarians with a focus on library instruction, 
and two teaching faculty from the 
Instructional Technology & Learning 
Sciences Department, all with a vested 
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interest in helping students improve their 
synthesis skills. The researchers developed 
an information synthesis lesson, which was 
implemented in four sections of an English 
2010 class. English 2010, Intennediate 
Writing: Research Writing in a Persuasive 
Mode, is a required second-year 
composition class. The focus of the class, 
according to the course description, is the 
“writing of reasoned academic argument 
supported with appropriately documented 
sources. [It] focuses on library and Internet 
research, evaluating and citing sources, oral 
presentations based on research, and 
collaboration.” Writing products were 
collected at the end of the intervention in 
addition to students’ final papers for the 
course. These products were analyzed using 
the synthesis rubric that was created and 
adapted from the previously mentioned 
rubrics. 

Participants 

The 87 study participants were enrolled in 
four Spring 2013 sections of English 2010, 
all taught by the same instructor. The 
enrollments for the four sections consisted 
of 21, 22, 23, and 21 students respectively. 
While biographical data on individual 
students was not collected, the total 
participant pool was 32 female students and 
55 male students. There were five freshmen, 
29 sophomores, 34 juniors, and 19 seniors. 
For the paper analysis, nine papers were 
randomly selected from each section. 

The control group participants were enrolled 
in four Fall 2011 sections of English 2010, 
all taught by the same instructor. There were 
89 total control group students, of whom 44 
were female and 45 were male. There were 
six freshmen, 26 sophomores, 37 juniors, 
and 20 seniors. 
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Measures 

To measure the extent to which infonnation 
synthesis took place in the written products 
(synthesis paragraphs and final papers), 
researchers in this study developed an 
information synthesis rubric based on an in- 
depth analysis of student papers and a series 
of infonnation literacy rubrics that included 
aspects of synthesis. The rubric consisted of 
five categories: A. Source variety, B. Uses 
information from sources effectively, C. 
Identifies conversations among information 
from different sources, D. Organizes 
sources overall in a meaningful, purposeful 
way, and E. Analyzes sources to create 
something new or draw conclusions and 
make generalizations. These categories were 
scored as unacceptable (0), needs 
improvement (1), developing (2), advanced/ 
mastery (3). The rubric was revised through 
three iterations in which researchers applied 
it to a total of ten student papers from the 
same course in a previous semester. This 
initial assessment process ensured 
consistency between raters in measuring 
students’ level of proficiency in 
synthesizing information, and resulted in the 
final rubric version in Appendix A. 

Procedures 

The information synthesis lesson was 
implemented in four sections of ENGL 
2010. The same researchers taught all four 
sections. The lesson (see Appendix B) 
lasted approximately 75 minutes and broke 
the synthesis process down into sequenced 
tasks. The lesson was based on an 
information synthesis workshop developed 
by Johnson at Arizona State University 
West (2003). Students were placed in 
groups of three. Using PowerPoint, the 
librarians provided instructions and 
discussed the characteristics of synthesis. 
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After basic definitions were established, the 
guided practice began. Each step of the 
guided practice had clear time delineations 
with corresponding instructions on a 
PowerPoint slide. Students were instructed 
to read two articles. Each student read one 
unique article and one article in common 
with the other members of his or her group. 
Students each highlighted the main points or 
areas about which they had questions. Then 
students wrote the five main points of their 
assigned articles on post-it notes, shared 
their notes with group members, and 
worked together to group main points by 
topic. Once students established general 
categories, they regrouped their post-it 
clusters and titled each one. At the end of 
the lesson, students were instructed to 
individually write a paragraph that 
synthesized one of their group’s clusters. 

Two researchers observed the lessons to 
record students’ reactions, paying special 
attention to student questions and their 
difficulties with particular steps in the 
process. After each lesson, the researchers 
asked for student and instructor feedback 
through debriefing and a short online 
survey. They collected students’ clusters of 
concepts derived from in-class readings for 
later analysis, and at the end of the semester, 
they collected students’ final papers as well. 

Results 

The researchers scored ah student papers 
using the rubric and tested their inter-rater 
reliability. They then measured whether the 
information synthesis intervention improved 
student synthesis skills, based on student 
papers and the synthesized paragraphs. 
Finally, the researchers established whether 
counting visible markers of synthesis could 
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predict the level of synthesis in student 
papers scored by the rubric. 

The information synthesis rubric 
To test the infonnation synthesis rubric for 
reliability, the researchers scored 72 final 
student papers (36 from the control group 
and 36 from the treatment group). Each 
paper was rated by two raters individually 
and then discussed to form a consensus. To 
test the reliability of the rubric, inter-rater 
reliability was calculated using Cronbach’s 
alpha. Raters had acceptable reliability 
overall (a = .72); the breakdown was a 
= .73 on control papers and a = .71 on 
treatment papers. Cronbach values between 
0.70 and 0.95 are considered to be 
acceptable (Tavakol & Dennick, 2011). 

The impact of the information 
synthesis lesson 

The researchers calculated the difference 
between the control and treatment group 
papers for each subscale. To determine if 
there was a significant difference between 
the calculated scores of the control and 
treatment papers, an analysis of variance 
was calculated for each of the five subscales 
of the rubric. Subscales A, B, D, and E had 
no significant differences. A main effect F 
(1, 70)=7.36, p<.01 was reported for 
Conversations Among Information from 
Different Sources (subscale C), indicating 
that the treatment group papers were better 
than the control group papers for this 
subgroup. Out of the five subscales, 
subscale C is essential, particularly in its 
relationship to category E, which focuses on 
students’ ability to enter the conversation in 
a meaningful way. In order to enter the 
conversation, students must first be able to 
identify and articulate the conversations 
already taking place, which is the focus of 
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Table 1—Analysis of variance for the rubric’s five subscales 


Subscale 

F 

p-value 

A - Source Variety 

0.31 

0.58 

B - Using infonnation from sources effectively 

1.40 

0.24 

C - Identifies conversations among infonnation from 
different sources 

7.36 

0.01* 

D - Organizes sources overall in a meaningful, 
purposeful way 

1.02 

0.32 

E - Analyzes sources to create something new or draw 
conclusions and make generalizations 

1.58 

0.21 


*Significant at p<.05. 


category C. Table 1 reports the analysis of 
variance for all five subscales. 

Synthesis Paragraph Analysis 
At the end of the intervention, each group 
had several infonnation clusters on their 
butcher paper (see figure 1). Every cluster 
contained several sticky notes, each note 
including a key point extracted from articles 
read by the students. The sticky notes were 
grouped together in topical clusters as part 
of an iterative group process. 

At the end of the lesson, student participants 
were asked to pick a cluster and write a 
paragraph synthesizing it. Students 
submitted their synthesis paragraphs using a 
web form. In the writing instructions for the 
synthesized paragraph (provided in class 
and also on the form), students were asked 
to use the article ID (e.g., A, B, C) to cite an 
article in the paragraph. The same web form 
also collected additional data: student name, 
group number, how many different articles 


were contained on the sticky notes in the 
cluster they used for writing their paragraph. 
Students submitted 74 synthesized 
paragraphs. After removal of paragraphs 
without in-text citations, paragraphs based 
on single-article clusters, and paragraphs 
from students who missed the intervention, 
a total of 52 paragraphs remained for the 
analysis. 

The 52 paragraphs were analyzed using 
subscale C of the synthesis rubric, which 
seemed to most robustly address the key 
skill of making connections between 
sources. No or very weak connections were 
made for 9 (17.3%) paragraphs; only 
implicit connections between the articles 
were made for 10 (19.2%) paragraphs; some 
explicit connections were made for 14 
(26.9%); and several explicit connections 
were made for many of the paragraphs (19 
or 36.5%). 

When comparing the level of synthesis in 
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Figure 1—Information from papers organized in named clusters 



the paragraphs with the level of synthesis 
found in student papers, the paragraphs 
fared much better in the highest score 
category of subscale C (Identifies 
conversations among information from 
different sources). This remained true after 
adding paragraphs without text citations 
back into the analysis (see Table 2). This 
finding suggests that our intervention does 
help students with the process of 
synthesizing at the paragraph level, but that 
the process does not necessarily transfer to 
the same extent to a larger scale project, 
such as an entire paper. 

Other methods for measuring 
synthesis 

After scoring each paper using the rubric, 
the researchers wanted to know if there 
were other identifiers of synthesis, such as 
how many times a student used a source 
across paragraphs. With this in mind, trends 
were identified using a counting analysis 
developed by the researchers. First, each 


source was highlighted and numbered in the 
works cited/bibliography list with a 
different color. Second, each source used in 
the paper was highlighted in the matching 
color. A source could have been used in the 
paper, but not listed in the reference list. 
These sources were counted and added to 
the works cited list, using the author or title, 
if available. Third, a comment was added 
next to each source noting if it was quoted 
or referenced or both. Fourth, the number of 
sources used in each paragraph was counted 
(i.e., source #2 was used in three different 
paragraphs). And fifth, the researchers 
counted how many paragraphs from each 
essay used one source, how many used two 
sources, and so on (see Table 3). 

General counting observations 
As can be seen from Table 3, the control 
and the treatment papers had the same 
percentage of paragraphs with the same 
number of sources. Over one-half (56%) of 
the paragraphs citied no sources, while 
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Table 2—Rubric Subscale C scores of student writing products 


Writing Products 

N 

Rubric 
Score CO 

Rubric 

Score 

Cl 

Rubric 

Score 

C2 

Rubric 

Score 

C3 

Student Papers (no 
intervention) 

36 

4(11.1%) 

22 (61.1%) 

10 (27.8%) 

0 (0%) 

Student Papers 
(intervention) 

36 

2 (5.6%) 

12 (33.3%) 

22 (61.1%) 

0 (0%) 

Paragraphs (intervention) 

52 

9(17.3%) 

10(19.2%) 

14 (26.9%) 

19 (36.5%) 

All Paragraphs* 
(intervention) 

71 

28 (39.4%) 

10(14.1%) 

14(19.7%) 

19 (26.8%) 


* All paragraphs include paragraphs based on single article clusters (3) and paragraphs without any in-text 
citations (16). All these 19 articles received a rubric score of 0. 


almost one-third (29%) cited only one 
source. However, having a large number of 
single-source paragraphs does not 
necessarily mean the paper scored low on 
synthesis overall, as evidenced by control 
group paper 046-12. This paper received 
one of the two highest synthesis rubric totals 
(score 10), but had nine paragraphs with 
only a single source in them. In contrast, the 
other high scorer on the synthesis rubric, 
paper 046-17 (score 10), had three 


paragraphs with two sources in them and 
one paragraph with four sources, which 
could indicate a high level of synthesis. 

Source use 

A common scenario for students was to 
include a number of sources in the 
references page that were not referenced or 
quoted in the paper. In this case, use refers 
to any time a source was referenced either 
through paraphrasing or by direct quoting. 


Table 3—Number of paragraphs with 0, l, 2, 3, and 4 or more 

SOURCES CITED 



%of 

paragraphs 
with 0 

sources 

%of 

paragraphs 
with 1 

source 

%of 

paragraphs 
with 2 

sources 

%of 

paragraphs 
with 3 

sources 

%of 

paragraphs 
with 4+ 

sources 

Total # 
Paragraphs 

Control 

56% 

29% 

11% 

2% 

2% 

9.916 

Treatment 

56% 

29% 

11% 

2% 

2% 

11.421 
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Table 4—Sources quoted vs referenced and used across 

MULTIPLE PARAGRAPHS 



Source 

Quoted 

Source 

Referenced 

used 
in 1 

used 
in 2 

used 
in 3 

used 
in 4 

used 
in 5+ 

Total # of 
Sources 

Control Total 

60 

167 

119 

29 

8 

0 

0 

156 

Treatment Total 

70 

233 

161 

34 

11 

3 

1 

210 


Most students only used a single source per 
paragraph; there were 119 instances of this 
for the control group and 161 instances for 
treatment group papers. The treatment group 
used 54 more sources total than the control 
group. 

Papers that used one source often-an 
instance of a source being used in three or 
more different paragraphs-tended to have 
lower synthesis scores (mode=5). From the 
control group, paper 049-20 with a rubric 
score of nine, and paper 046-12 with a score 
of 10 were the exceptions. These students 
tended to use one source frequently, but 
they had higher synthesis scores. From the 
students in the treatment group, most papers 
using one source in three or more 
paragraphs scored a six or seven from the 
rubric. 

In general, students paraphrased their 
sources much more often than quoting. In 
total, students from the control group quoted 
their sources 60 (26%) times and 

paraphrased them 167 (74%) times, while 
students with the treatment group quoted 70 
(23%) times and paraphrased 233 (77%) 
times (see Table 4). In tenns of infonnation 
synthesis, this indicates that students from 
the treatment group are using multiple 
sources in their papers, and they use the 
information from those sources in a more 


advanced way, paraphrasing instead of 
directly quoting. These two areas are 
covered by subscale A and B in the rubric. 

Comparing the synthesis rubric and 
the counting analysis 
While the number of sources per paragraph 
did not necessarily predict a high or low 
score on the synthesis rubric as a whole, 
there was a connection between papers that 
scored a two or higher in subscale C from 
the rubric; they all had at least one 
paragraph with two sources. 

There is a higher correlation between the 
counting analysis and the subscale C 
consensus score for the treatment papers 
(r=0.74) versus the control papers (r=0.63). 
Occasionally higher scores had more 
sources used, however there were also 
instances where there was no correlation 
between high rubric scores and high number 
of sources used. For example, one of the 
papers only used three sources and scored 
an 8 on the rubric. From the treatment 
group, another paper used four sources and 
scored an 8 on the synthesis rubric. This 
indicates that number of sources used does 
not necessarily correlate with a student’s 
ability to synthesize that infonnation. 
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Discussion 

All three assessment techniques-the rubric 
assessment of the paragraphs from the 
intervention, the rubric assessment of the 
final papers in comparison to the control 
group, and the counting analysis of how 
sources were used within final 
papers-indicate students struggle with 
synthesis and benefit from teaching methods 
that break down the different skills involved 
in synthesis. Each of these techniques 
revealed key student behaviors that can help 
librarians and educators modify their 
instruction practices. 

These findings confirm the difficulty of the 
synthesis process for students, detailed in 
much of the previous literature (Howard, 
Serviss, & Rodrigue, 2010). The findings 
also show that teaching interventions in this 
process can help students improve their 
ability to synthesize. In this case, the 
intervention slowed down this process for 
students, and it helped students identify 
main ideas and collaboratively group those 
ideas with others. Interestingly, the skill of 
being able to identify conversations among 
different texts appeared to transfer to 
students’ final papers. Students may have 
also benefited from having explicit 
definitions of synthesis, such as sharing and 
discussing the specific elements of the 
comprehensive synthesis rubric used in this 
study. 

Defining Synthesis & Rubric Creation 
As detailed in the literature, definitions of 
what synthesis is and what it looks like vary 
widely. One of the major components of this 
research emerged with the realization that 
solid markers for measuring synthesis are 
difficult to find, largely due to its 
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complexity and scope. The creation of this 
rubric helped to pinpoint what synthesis 
involves and what it looks like in a paper. 
The researchers determined that five 
categories are most significant when 
identifying the presence of information 
synthesis: mainly source variety (subscale 
A), using information sources effectively 
(subscale B), identifying conversations 
among sources (subscale C), organizing 
sources meaningfully (subscale D), and 
analyzing sources to create something new 
(subscale E). These categories and their 
descriptions at each level vary from other 
existing rubrics, although they are adapted 
from pieces of existing ones. 

During the rubric creation and revision 
phase, the researchers continuously reached 
for measurable language whenever possible. 
For example, source variety (subscale A) 
clarifies numerically how varied the sources 
must be to be categorized at each level, 
focusing particularly on whether the student 
uses sources that speak to a variety of 
perspectives rather than using sources that 
all trend towards the same view. Subscale 
B, using information sources effectively, 
attempts to distinguish haphazard use of 
sources from summarizing main ideas. 
Subscale C, identifying conversations, 
underwent the most revision during this 
process, including defining “explicit” as 
sources that included textual indicators (i.e., 
using terms such as “likewise” or “in 
contrast”), or the sources are placed side by 
side in the same paragraph. Subscale D 
attempts to show variation in the 
effectiveness of the organization of sources. 
Subscale E differentiates between works 
that make assumed generalizations and 
those that come to well-reasoned logical 
conclusions as a result of how they use and 
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respond to their sources. 

These carefully articulated categories, or 
elements of synthesis, combined with 
measurable and specific descriptions at each 
level, helped the researchers to feel 
confident in how they were assessing 
students’ ability to synthesize. The 
categories also helped the researchers to 
better understand where an instructor or 
librarian might intervene in a student’s 
process and how to more carefully scaffold 
activities that support students in areas in 
which they struggle. The results of the inter¬ 
rater reliability analysis reinforced 
confidence in the reliability of the rubric, as 
well as in the usefulness of the intervention. 

Impact of the intervention on the 
synthesis paragraphs 
Judging by the synthesized paragraphs that 
students wrote directly after the intervention 
(scored on subscale C of the rubric), it 
appears that the synthesis lesson helped 
students with the process of synthesizing 
information from difference sources into 
paragraphs. This effect is much less clear 
when looking at the final student papers. 
One of the reasons for this could be that the 
intervention combined individual thought 
processes with group discussions, while the 
final papers are individual efforts. Another 
reason could be that the paragraphs were 
written immediately after the intervention, 
while the papers were not completed for 
another two weeks. Additionally, several 
students had already started on their final 
papers before our intervention; they may 
have been too far along in the writing 
process to start over and fully apply the 
synthesis process they learned in class. 

Another aspect to consider is suggested by 
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Cavanaugh and Perlmutter (1982) who 
noted that students tend to use strategies 
when instructed to do so, but stop using 
these strategies when they are no longer 
required. Students in this study went 
through the synthesis strategy as part of the 
intervention and just before they wrote their 
synthesis paragraphs. The instructions for 
the final paper did not mention that students 
should use the synthesis strategy. There are 
two possible solutions for this in future 
work. One would be to slowly fade the 
scaffold, which would lengthen the 
intervention. The other would be to create a 
synthesis tool that would serve as a 
cognitive support with the synthesis steps 
built in across the entire research and 
writing process. Pea (2004) calls this latter 
approach a case of distributed intelligence, 
where the intelligence is incorporated into 
the software. Changing the intervention to 
allow fading the scaffold should be possible 
by using an infonned learning approach 
(Maybee, Bruce, Lupton, Rebmann, 2013; 
Bruce and Hughes, 2010), where students 
actively use information to learn instead of 
being taught an infonnation skill like 
synthesis in isolation. The researchers are 
currently experimenting with this approach 
in an online class where students curate 
content to create learning modules. 

The impact of the intervention on 
final papers 

The majority of the control group papers 
(from the Fall class that did not receive the 
intervention) received a score of one on 
subscale C, while the majority of the 
treatment group papers submitted after the 
intervention received a score of two. This 
indicates that our intervention had an effect. 
Interestingly, the synthesis scores for 
submitted paragraphs are more evenly 
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distributed across all four categories in each 
subscale compared to the final paper scores, 
which are mostly clustered in the two 
middle scores. Conversely, papers that used 
three or four sources in a paragraph did not 
necessarily correlate to having higher rubric 
scores. 

Effectiveness of Counting Analysis 
The counting method did reveal some 
interesting patterns about how students use 
sources. There does not appear to be a 
correlation between the number of 
paragraphs containing certain numbers of 
cited sources and the total synthesis rubric 
score. This might be because the individual 
subscales are measuring something different 
that cannot be captured by counting. 

Also, when looking at how many times 
sources were used across the submitted 
paragraphs, there was a significant 
difference between the control and the 
experimental group on subscale C. The 
rubric scores from the other categories did 
not appear to relate significantly to the 
counting analysis results. Subscale C did 
correlate well with the rubric scoring 
because the language of the rubric clarified 
that papers that received a two in subscale C 
had to have at least two sources in one 
paragraph in conversation with one another. 

While this analysis was fairly limited in 
correlating numbers of sources to 
identifying synthesis, more research is 
needed to explore alternative ways of 
measuring this skill. In general, this analysis 
did help identify student behaviors relating 
to synthesis. There were also quite a few 
instances of sources listed in a reference 
page that were not actually used in an 
identifiable way in the paper. This may 
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reflect a tendency for student to place more 
emphasis on finding and documenting 
sources, rather than learning to use them 
effectively to support their argument and 
contribute something new to a conversation. 
These findings indicate a continued need for 
teaching synthesis, but also for revising 
scaffolds, such as the intervention used in 
this study, to target some of these specific 
areas. 

Conclusion 

Despite the difficulty of learning and 
teaching synthesis, breaking down the 
processes involved in synthesis, both in 
definition and in scaffolded practice, can 
help instructors teach and assess synthesis 
more effectively. In this study, the authors 
created a comprehensive, reliable rubric that 
clearly defined multiple aspects of synthesis 
and helped to determine which elements of 
synthesis students struggle with, as well as 
identifying when synthesis was employed in 
student work. Using this rubric, the 
researchers detennined that the attempts to 
break down the synthesis process in the 
intervention may have improved students’ 
ability to identify conversations within the 
literature and incorporate those 
conversations, in identifiable ways, into 
their final persuasive research papers. 

These findings are relevant for instructors 
and for librarians as they support the 
research process. The issue of whether 
librarians should be involved in teaching 
synthesis is an important one. As librarian 
roles continue to shift, and as they 
collaborate with faculty to find ways to help 
students with the threshold concepts in the 
Framework for Information Literacy for 
Higher Education, librarians may be 
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increasingly more involved in teaching 
higher level concepts like synthesis. In this 
study, a librarian taught the intervention, but 
the lesson plan can also be shared by 
librarians with instructors looking for ways 
to help students learn synthesis. It is clear 
from previous studies on student research 
skills, such as reports from Project 
Information Literacy (Head & Eisenberg, 
2009), that appropriate usage of information 
is a major area of difficulty for students. 
Students and instructors may not recognize 
that librarians can assist the synthesis 
process, but as this study shows, they can. 
Librarians must be clearer about their role in 
helping with any facet of the research 
process, including synthesis. 

Future avenues for research include 
adapting and creating more interventions 
that breakdown the synthesis process, 
particularly in relation to the five categories 
identified in the rubric used for the present 
study. In this case, the intervention targeted 
identifying conversations, but more research 
is needed in how to help students with the 
other rubric elements. Further inquiry into 
using the informed learning approach may 
also help identify effective ways to scaffold 
elements of synthesis. Alternative ways to 
assess synthesis, such as the counting 
analysis method attempted here, also 
warrants more exploration. Methods like 
these can help librarians and instructors 
identify when synthesis is occurring and 
what it looks like in order to teach students 
effective strategies for synthesizing 
information. 
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Appendix A—Synthesis rubric 


Definition from 

1 lesson _| 

Not present or 
unacceptable = 0 

Needs Improvement 
= 1 

Developing = 2 

Advanced/Mastery = 3 

A. Source Variety 

Does not use 
multiple sources 
(<2). Sources do 
not need to be in a 
reference list; any 
mention of any 
outside source 
works. 

Uses a few different 
sources, but with little 
variation (1 source of 
variation) in 
perspective, trending 
toward the same 
view. Sources do not 
need to be in a 
reference list; any 
mention of any 
outside source works. 

Uses a variety of 
sources that cover some 
of the differing 
perspectives (2 or more 
perspectives). Sources 
do not need to be in a 
reference list; any 
mention of any outside 
source works. 

Uses multiple sources which 
address multiple 
perspectives, including 
opposing viewpoints. 

Sources do not need to be in 
a reference list; any mention 
of any outside source works. 

B. Using 

information from 

sources 

effectively 

Does not present 
information from 
sources. No in-text 
citations present. 

Uses information 
from sources with no 
added value, with 
little or no summary - 
used only as support 
(haphazard, provide 
too much or too little 
infonnation, and/or 
serve no clear 
purpose). 

Uses information from 
sources through direct 
quotes and/or 
paraphrasing and begins 
to summarize main 
ideas (making main 
ideas more clear and 
succinct; implicit 
connections to the thesis 
or main ideas of the 
paper). 

Uses information from 
sources through direct quotes 
when necessary, 
paraphrasing, summarizing, 
and explicit connections 
(making the main ideas clear, 
succinct, and connected to 
the thesis or main ideas in 
the paper). 

C. Identifies 

conversations 

among 

information from 

different sources 

No or very weak 
connections are 
made between 
sources. The 
reader cannot see 
how the sources 
are related to each 
other. 

Few implicit 
connections are made 
between sources. 
Similarities, 
differences, 
relationships and 
patterns are rarely 
identified so it is 
difficult for the reader 
to see how the 
sources are related. 

There are some explicit 
connections between 
sources (textual 
indicators or side by 
side in paragraph 
connections). 

Similarities, differences, 
relationships, and 
patterns are sometimes, 
but not consistently, 
identified so the reader 
can see how some of the 
sources are related. 

There are several explicit 
connections between sources 
including connections 
between contradictory 
sources. Similarities, 
differences, relationships, 
and patterns are almost 
always identified so the 
reader can see how the 
sources are related and how 
they support the thesis. 

D. Organizes 
sources overall in 
a meaningful, 
purposeful way 

Does not use any 
information from 
sources and lacks 
organization. 

Uses information 
from sources and 
attempts to organize 
infonnation but the 
organization is not 
effective in revealing 
important patterns, 
differences or 
similarities. (AACU) 

Uses information from 
sources and effectively 
organizes information to 
reveal some important 
patterns, differences or 
similarities to focus. 
(AACU) 

Uses information from 
sources and effectively 
organizes information to 
reveal insightful patterns, 
differences, or similarities 
related to focus. (AACU) 

E. Analyzes 
sources to create 
something new or 
draw conclusions 
and make 
generalizations 

No conclusions or 
generalizations 
offered 
(Evergreen). 

Author does not 
attempt to make 
sense of the 
infonnation he/she 

uses. 

The relationship 
between the ideas in 
sources and the 
author's conclusions 
or implicit 
generalizations are 
assumed or unclear. 

The author uses 
infonnation sources to 

come to some 
reasonable 

generalizations or well- 
reasoned conclusions. 
(Ford, T.) 

The author relates 
knowledge from several 
areas/sources of information 
in order to demonstrate 
comprehension, make 
insightful analyses, and draw 
clear conclusions. 
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Appendix B—Detailed Lesson Plan: Teaching Information 
Synthesis (75 minutes) 

Learning Outcomes 
Students will be able to: 

1. Analyze texts in order to identify important facts, concepts, and questions for further research. 

2. Consider the ideas from multiple sources of information in order to infer relationships between concepts, facts, and 
arguments. 

3. Organize ideas from multiple sources in order to create meaningful groups of related concepts. 

4. Restate facts and concepts in their own words in order to communicate a new understanding of the relationships 
between ideas from multiple sources. 

I. Introduction (5 minutes) 

1. What we’re doing today and why (opportunity to opt out of being part of the study) 

2. Final product: graded, posted synthesized paragraph 

3. Synthesis Defined 

II. Guided Practice: 

Context for Today (1 minute): 

“One of your family members has been diagnosed with depression. You know very little about depression, the possible 
causes or treatment options. You decide to do a little background reading to familiarize yourself with this topic so 
you can provide better support.” 

2. Have students form teams of 3 to read articles. (8 minutes) 

a. Individually they will read 2 short articles - 1 shared, 1 distinct 

b. Students will highlight the following things: 

i. What seems important or meaningful 

ii. Points they could explain, share 

iii. Points that need clarification 

3. Have students share what they learned as a Group (10 minutes) 

a. Share what they learned 

b. Ask questions for understanding 

c. Teach each other 

4. Organizing Infonnation Individually (8 minutes) 

a. Individual clustering using post-its 

i. Pull out key points - highlighted parts 

ii. Put article letter in comer of post-it (A or B) 

iii. Five post-its/main ideas per article 

iv. One idea per post-it (about 3-5 words); use large letters 

5. Organizing Infonnation as a Group (10 minutes) 

a. As a team, group like post-its together 

i. Start with a few, add a few more 
ii. Work until all post-its are in a cluster 

b. As a team, name the clusters 

i. 1 or 2 word nickname 

ii. Must fit EVERY post-it in the cluster 

6. Reviewing your information as a Group (5 minutes) 

a. Narrowing (using big clusters as main themes, splitting clusters) 

b. Broadening (drop clusters out as too minor, return to articles to get more info., return to 
gathering phase to find more articles) 

III. Wrap-up & Making Connections as a Group (8 minutes) 

a. Creating an Outline 

b. Avoiding Plagiarism 

IV. In-Class Assignment Individually (20 minutes) 

a. Put articles aside. 

b. Students will post in Canvas before class ends one paragraph using one cluster of main ideas 
from their organized post-its. Students can use in-text citations for articles A, B, etc. where needed 
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